Shubhangi
Tandon
Mananger, Applied Research 2 at eBay inc
"Whose Similarity Is It Anyway? Persona-Aware Evaluation for Visual Search"
"Whose Similarity Is It Anyway? Persona-Aware Evaluation for Visual Search"
Don’t miss out and join visionaries, innovators, and thought leaders from all over the world at the Women in Tech Global Conference.
Vote by Sharing
Unite 100 000 Women in Tech to Drive Change with Purpose and Impact.
Do you want to see this session? Help increase the sharing count and the session visibility. Sessions with +10 votes will be available to career ticket holders.
Please note that it might take some time until your share & vote is reflected.
Session: Whose Similarity Is It Anyway? Persona-Aware Evaluation for Visual Search
Visual similarity is inherently subjective: a sneaker collector, casual shopper, and teenager may interpret the same image pair in very different ways. Yet, most visual search systems rely on one-size-fits-all evaluation, overlooking the nuances of user expertise, purpose, and aesthetic preference. This talk presents a persona-aware evaluation framework that leverages Large Language Models (LLMs) to simulate user-specific perspectives. We design four distinct personas—Fashion Expert, General Consumer, Tech Evaluator, and Teen User—each assigning relevance scores (1–4) to query–result image pairs, accompanied by commentary. By framing prompts through persona-driven bias, our approach moves beyond generic classification, yielding diverse training labels from a single image pair and uncovering subjectivity-aware failure cases. The framework ensures repeatability, scalability, and interpretability for a task traditionally viewed as subjective. Applied to domains like collectibles, fashion, and electronics, it provides a pathway toward user-segment–aware evaluation and ultimately enables personalization in multimodal visual search systems.Key Takeaways
- Persona-aware evaluation with LLMs provides a scalable way to simulate diverse user perspectives in scoring visual relevance.
- Persona-driven prompting introduces controlled bias, capturing inter-persona variance as a useful signal rather than noise.
- Toward personalization: this approach lays the foundation for user-segment–aware and eventually individual-level multimodal search systems
Bio
Shubhangi Tandon is Manager of Applied Research at eBay, where she leads the Content Understanding team driving innovations in Visual Shopping, Generative AI, and multimodal retrieval. She has over ten years of experience advancing large-scale search and recommendation systems, including building eBay’s first embedding-based retrieval models and launching GenAI-powered search and selling experiences that reach millions of users.Her work spans computer vision, natural language processing, and multimodal modeling, with multiple patents in visual search, retrieval, and navigation. She has published at top venues including IEEE Big Data and SIGDIAL, and holds an M.S. in Computer Science from the University of California, Santa Cruz, and a B.E. in Information Technology from Delhi College of Engineering.
